As a method of increasing human-friendliness by giving a robot an appearance similar to a human or animal (hereinafter referred to as a “human or the like”), a method of substituting an image of a human or the like for an image of a robot using technology of mixed reality is considered.
In this method, it is possible to reduce the cost or time necessary to design/manufacture the robot because it is unnecessary to make the appearance of the robot similar to the appearance of a human or the like when the robot is manufactured. In addition, the appearance can also be easily changed by changing image data to be synthesized.
Here, it can be viewed as if there is a human or the like even in a space in which there is no human or the like by synthesizing a virtual human or the like with a real image without using the robot. However, in this method, finding a positional relationship of a real object with the virtual human or the like becomes problematic.
For example, when an observer (a person who views a synthesized image) has held out his/her hand toward the position of the virtual human or the like, it is necessary to control a display/non-display of a virtual human or the like in an overlapping portion so as to represent the front-rear relation of overlapping between the hand viewed by the observer and the virtual human or the like. Also, hereinafter, the front-rear relation of overlapping objects (including a virtual object) when viewed by the observer is referred to as a “masking relationship.”
For example, a time of flight (TOF) method is a method of measuring a position of a real object (a distance from a point in view of the observer) such as the observer's hand so as to find this masking relationship. In the TOF method, light is radiated from a light source to a subject (a real object such as the observer's hand) and reflected light is detected by an optical sensor. Then, based on a time until the reflected light is detected after the light is radiated (TOF of the light), a distance between the light source or sensor and the subject is measured.
However, when the masking relationship is found using the TOF method, a precise device for performing the TOF method becomes necessary and the manufacturing cost of the device increases.
On the other hand, in the above-described method of substituting the image of the human or the like for the image of the robot (real object), the position of the robot shows a position of the virtual human or the like. Accordingly, it is possible to find the front-rear relation of the virtual human or the like and the observer's hand by finding the front-rear relation of the robot and the observer's hand, and a device or the like for performing the TOF method is unnecessary.
In addition, in the above-described method of substituting the image of the human or the like for the image of the robot, it is possible to provide a tactile sensation when the observer is in contact with the robot.
However, in the above-described method of substituting the image of the human or the like for the image of the robot, processing of different parts of a shape of the image of the robot and a shape of the image of the virtual human or the like becomes problematic. This point will be described with reference to FIGS. 11 to 13.
FIG. 11 is an explanatory diagram illustrating an example of a real image, and includes an image I1101 of a user's hand and an image I1102 of a robot. In addition, FIG. 12 is an explanatory diagram illustrating a human image (for example, an example of computer graphics (CG)) to be synthesized. In addition, FIG. 13 is an explanatory diagram illustrating an example of an image obtained from a synthesis result. The image of FIG. 13 is synthesized by substituting the human image of FIG. 12 for a region portion of the image I1102 of the robot using chroma key technology in the real image of FIG. 11.
In FIG. 13, a portion protruding from the region of the image I1102 of the robot is clipped from the human image according to a difference between a shape of the image I1102 (FIG. 11) of the robot and a shape of the human image (FIG. 12). An uncomfortable feeling is given to the observer according to this partial clipping of the human image. In addition, when the region of the image I1102 of the robot is larger than the region of the human image in contrast to the example of FIG. 13, a portion other than the human image of the image of FIG. 12 is included in a synthesized image. For example, if the portion other than the human image is masked in the image of FIG. 12, image information of a protrusion region of the image I1102 of the robot is omitted in the synthesized image. According to the incorporation of the portion other than the human image (for example, the omission of the image information), the uncomfortable feeling is given to the observer as in the case in which the human image is partially clipped.
As one countermeasure for this problem, there is a method using an image processing system disclosed in Patent Literature 1. The image processing system disclosed in Patent Literature 1 includes an image selection device for selecting CG of a form of a real object and a form suitable for a positional relationship between the observer who views the real object and the real object, a halo generation device for generating a halo image around the CG, a synthesized image generation device for generating a synthesized image of the CG and the halo image, and an image display processing device for displaying the synthesized image on a display viewed by the observer so that the synthesized image is superimposed on the real object.
When the image of the virtual human or the like is substituted for the image of the robot in the real image using the image processing system, the size of the image of the human or the like is configured to be slightly less than the size of the image of the robot. Then, the halo image is displayed in the synthesized image for a region of the image of the robot protruding from the image of the human or the like and the uncomfortable feeling for the observer can be reduced.